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development and evaluation of student and teacher materials for an integrated 
3-year high school mathematical sciences curriculum for all students. This 
paper provides a brief overview of the CPMP curriculum in terms of its design 
and theoretical framework as well as a profile of the mathematical 
achievement outcomes of students who participated in the national field test 
of the curriculum. Each CPMP course was field tested in 36 high schools. 
Students in CPMP courses after 1 year, 2 years, and 3 years illustrated 
better understanding of and ability to reason in quantitative situations than 
did students in more traditional mathematics courses and in the nationally 
representative norm group. The findings reported in this study are likely to 
be the result of CPMP students engaging frequently in class, homework, and 
assessment activities that provide them ample opportunity to cogitate on 
problems presented in realistic contexts about mathematical models for those 
problems, and seeing the connections among and patterns in various 
representations of those models. Contains 41 references. (ASK) 
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An Emerging Profile of the Mathematical Achievement 
of Students in the Core-Plus Mathematics Project 

Central to all the policy reports spearheading this decade of reform (American 
Association for the Advancement of Science [AAAS], 1989; Mathematical Sciences Education 
Board [MSEB], 1990; National Council of Teachers of Mathematics [NCTM], 1989; (National 
Research Council [NCR], 1989) is a commitment to the belief that all students can learn 
mathematics and to the objective that all students must learn more, and different, mathematics 
than in the past. History has shown that appropriate instructional materials are essential if 
.recommendations for school mathematics reform are to be implemented (Begle, 1973; Usiskin, 
1985). Recognizing this need, the National Science Foundation, in the early 1990s, awarded 
multi-year grants to 13 elementary school, middle grades, and high school projects (Education 
Development Center, 1998) to design, evaluate, and disseminate innovative curricula that 
interpret and implement the recommendations of the National Council of Teachers of 
Mathematics’ Curriculum and Evaluation Standards for School Mathematics (NCTM, 1989) and 
Professional Standards for Teaching Mathematics (NCTM, 1991). 

The Core-Plus Mathematics Project (CPMP) is one of four comprehensive high school 
curriculum projects that were funded by NSF in 1992. CPMP has completed development and 
evaluation of student and teacher materials for an integrated three-year high school mathematical 
sciences curriculum for all students. The completed curriculum is published under the title 
Contemporary Mathematics in Context (Coxford, Fey, Hirsch, Schoen, Burrill, Hart, Watkins, 
Messenger, & Ritsema, 1998). Research and development are currently in progress on a flexible 
fourth-year course continuing the preparation of students for college mathematics. 

PURPOSE 

In this paper, we provide a brief overview of the CPMP curriculum in terms of its design 
and theoretical framework and a profile of the mathematical achievement outcomes of students 
who participated in the national field test of the curriculum. The emphasis here is on large scale 
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quantitative achievement test results that may be of particular interest to educational policy 
makers. More focused research studies conducted in CPMP classrooms are reported elsewhere 
(cf. Flowers, 1995; Wilson & Lloyd, 1995; Kett, 1997; Lloyd & Wilson, 1997; Truitt, 1998). 
These references and others are included in the bibliography of CPMP publications appended to 
this paper. 

The profile of achievement presented in this paper is an “emerging profile” in that the 
CPMP curriculum remains under development and its evaluation will continue for several more 
years. In Spring 1998, Course 3 is being revised and edited for publication on the basis of the 
1996-97 field test. Selected data from the field test are still being processed and analyzed. Many 
of the Course 3 field-test students, now mostly seniors, have completed the ACT and/or SAT. 
That data is being collected and will eventually comprise an important part of the profile of 
CPMP student achievement. In addition. Course 4 is being pilot tested in 1997-98 and will be 
field tested the following year. The Course 4 field-test students will complete achievement 
measures that include college mathematics placement tests, which also will be an important part 
of the achievement profile. Thus, the achievement profile presented in this paper is based on 
much of the achievement data gathered during the first three years of the CPMP field test, but it 
does not include some important results that are still being compiled. 

BACKGROUND 

Development of the CPMP curriculum is based on several principles shaped by the 
practice of mathematics today, by research on teaching and learning, and by emerging 
technology. 

( 1 ) Mathematics is a vibrant and broadly useful subject to be explored and understood as an 
active science of patterns (Steen, 1990). 

(2) Each part of the curriculum should be justified on its own merits (MSEB, 1990). 

(3) Computers and calculators have changed not only what mathematics is imp ortant but 
also how mathematics should be taught (Zom, 1987; Hembree & Dessart, 1992; 
Dunham & Dick, 1994). 
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(4) Problems provide a rich context for developing student understanding of mathematics 
(Schoenfeld, 1988; Schoenfeld, 1992; Hiebert, Carpenter, Fennema, Fuson, Human, 
Murray, Olivier & Weame, 1996). 

(5) Deep understanding of mathematical ideas includes connections among related 
concepts and procedures, both within mathematics and to the real world (Skemp, 1987). 

(6) Classroom cultures of sense-making shape students understanding of the nature of 
mathematics as well as the ways in which they can use the mathematics they have 
learned (Resnick, 1987; Resnick, 1988; Lave, Smith, & Butler, 1988). 

(7) Social interaction (Cobb, 1995) and communication (Silver, 1996) play vital roles in the 
construction of mathematical ideas. 

(8) Small-group cooperative learning environments encourage more female participation in 
the mathematics classroom (Wisconsin Center for Education Research, 1994), and 
encourage a variety of social skills that appear particularly conducive to the learning 
styles of females and underrepresented minorities (Oakes, 1990; Leder, 1992). 

Curriculum Overview 

In developing the CPMP three-year core curriculum, we employed a “zero-based” 
process in which the inclusion of a topic was based on its own merits. In particular, in designing 
a particular course, we always asked; “If this course is the last mathematics students will have 
the opportunity to leam, is the most important mathematics included?” The fourth-year course is 
being designed to provide a smooth transition to collegiate mathematics. 

The CPMP four-year curriculum builds upon the theme of mathematics as sense-making. 
Investigations of real life contexts lead to (re)invention of important mathematics that makes 
sense to students and that, in mm, enables them to make sense of new situations and problems. 
Throughout it acknowledges, values, and extends the informal knowledge of data, shape, change, 
and chance that students bring to situations and problems. Each course in the CPMP curriculum 
features interwoven strands of algebra and functions, geometry and trigonometry, statistics and 
probability, and discrete mathematics. Mathematical modeling and its related concepts of data 
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collection, representation, interpretation, prediction, and simulation serve as an organizing 
principle for each of the strands. 

These four strands are connected within instructional units by common topics such as: 
sy mm etry, functions, matrices, and data analysis and curve-fitting. The strands also are 
connected across units by mathematical habits of mind such as: visual thinking, recursive 
thinking, searching for and describing patterns, making and checking conjectures, reasoning with 
multiple representations, inventing mathematics, and providing convincing arguments. The 
strands are unified further by the fundamental themes of data, representation, shape, and change. 

Each course in the CPMP core curriculum consists of seven units, each comprised of 
three to five multi-day lessons centered on big ideas, and a thematic capstone which enables 
students to pull together and apply the important mathematical concepts and methods developed 
in the course. Numerical, graphics, and programming/link capabilities of graphic calculators are 
assumed and capitalized on throughout each course. Use of this technology supports the 
emphasis of the curriculum and instruction on multiple representation (numeric, graphic, 
symbolic) and on goals in which mathematical thinking is central. 

The instructional materials are designed to promote a four-phase cycle of classroom 
activities, described below, designed to engage students in investigating and making sense of 
problem situations, in constructing important mathematical concepts and methods, and in 
co mmuni cating orally and in writing their thinking and the results of their efforts. Most 
classroom activities are designed to be completed by students working together collaboratively in 
heterogeneous groupings of two to four students. 

Each lesson is launched with a situation and related questions to think about which sets 
the context for the student work to follow. In the second or explore phase, students investigate 
more focused problems and questions related to the launch situation. This investigative work is 
followed by a class discussion in which students summarize mathematical ideas developed in 
their groups, providing an opportunity to construct a shared understanding of important concepts, 
methods, and approaches. Finally, students are given a task to complete on theirown, assessing 
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their initial understanding of the concepts and methods. Each lesson also includes tasks, 
intended primarily for out-of-class work, to engage students in modeling with, organizing, 
reflecting on, and extending their mathematical understanding. The CPMP curriculum and 
instructional model (Hirsch, Coxford, Fey & Schoen, 1995; Schoen, Bean & Ziebarth, 1996; 
Hirsch & Coxford, 1997) and professional development programs for teachers (Van Zoest & 
Ritsema, 1998) are described in more detail elsewhere. 

METHOD 

Sample 

Each CPMP course was field tested in 36 high schools in Alaska, California, Colorado, 
Georgia, Idaho, Iowa, Kentucky, Michigan, Ohio, South Carolina, and Texas. A broad cross- 
section of students from urban, suburban, and rural communities with ethnic and cultural 
diversity is represented. Because of difficulties of interpretation of tests given at different times 
in the school year, data from three semester-block schools was analyzed separately. This report 
focuses on achievement results for the 33 field test schools who were on a regular two-semester 
schedule. 

The field test schools were encouraged to include students with a wide range of 
achievement and interest in mathematics, and, where possible, students were grouped 
heterogeneously. Limitations at local sites did not always make this possible. Course 1 teachers’ 
descriptions of their entering CPMP students are summarized in Table 1. 



Table 1. ■ 

Percent of Teachers Giving Various Description of Their Field Test Students Upon Entering 
Course 1 



Description 


Percent 


No grouping - full range of ninth-grade students 


21.5 


Wide range of prior achievement but excluding best students 


43.0 


Wide range of prior achievement but excluding best and weakest students 


12.7 


More or less the typical Algebra 1 group 


15.2 


More or less the typical general mathematics group 


7.6 
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About one-fifth of the teachers reported that their classes included the full range of ninth- 
grade students. The most common CPMP class (as reported by 43.0% of the teachers) was 
comprised of students with a wide range of prior mathematics achievement and interest. Often, 
however, honors students were not included because they completed the grade nine course in 
eighth grade and moved on to a tenth-grade mathematics course in grade nine. Thus, the CPMP 
field test sample, as reported by the teachers, included students with a wide range of prior 
achievement and interest in mathematics, but honors or accelerated students are probably 
underrepresented. 

Traditional Comparison Classes At the beginning of Course 1, eleven field test schools 
volunteered to pretest and posttest students in traditional, comparison classes. The comparison 
classes were comprised of 20 algebra 1, five pre-algebra, three general mathematics, and two 
honors geometry ninth-grade classes. The nature of the instruction in the comparison classes was 
not specified in advance, but at the end of the year comparison teachers described what 
transpired. For example, a variety of traditional textbooks were used. Small group work was 
reported to be used either not at all or less than once a week by about 80% of the comparison 
teachers. About 74% of the comparison teachers reported that their students used a calculator 
more than once per week, although there is no data about how it was used. Solving linear 
equations in one variable was the main instructional goal for an average of 23% of the class time 
for the year, with up to 50% of the time spent on this topic in some algebra I classes. 

The Course 2 comparison group consisted of all students who were in the Course 1 
comparison group, completed a traditional sophomore mathematics class, and completed the 
Course 2 posttests. Only five of the 11 schools who had comparison groups in Course 1 were 
able to maintain them in Course 2. The main reason for this drop in number is that the Course 1 
comparison students enrolled in a variety of mathematics classes in their sophomore year and 
were difficult to locate and posttest at the end of the year. By the end of Course 3, the number of 
comparison students from the original pretested group that were available for posttesting was so 
small that a Course 3 comparison group was not feasible. 
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Instruments to Assess Achievement 

Standardized Tests One measure of mathematics achievement used in the CPMP field 
test is a standardized test called Ability to Do Quantitative Thinking (ATDQT), which is the 
mathematics subtest of the Iowa Tests of Educational Development (TIED) (Feldt, Forsyth, 
Ansley & Alnot, 1993). The ITED is a nationally standardized battery of high school tests 
developed by the Iowa Testing Programs, the same group that writes the widely used elementary 
school level Iowa Tests of Basic Skills (1TBS). The ATDQT is a 40-item multiple-choice test 
with the primary objective of measuring students’ ability to employ appropriate mathematical 
reasoning in situations requiring the interpretation of numerical data and charts or graphs that 
represent information related to business, social and political issues, medicine, and science. The 
ATDQT correlates highly with other well-known measures of mathematical achievement. 
According to research conducted by the test’s developers, correlation of the ATDQT, when given 
in grade nine, with the ITBS Mathematics total score in grade eight is .81; with students' final 
cumulative high school grade point average in mathematics courses is .59; with the ACT 
Mathematics test is .84; and with the SAT Mathematics test is .82. The ACT and SAT are 
usually completed in eleventh or twelfth grade. 

In addition to the ATDQT, a posttest comprised of released multiple-choice items from 
the 1990 or 1992 administration of the National Assessment of Educational Progress (NAEP) in 
twelfth-grade mathematics was administered at the end of Course 3. Items were chosen that 
measured outcomes that were of interest and provided a balance across the NAEP content 
(numbers & operations; measurement; geometry; data analysis, statistics & probability; algebra 
& functions) and process dimensions (concepts, problem solving, procedures). 

Performance Assessment Instruments The CPMP evaluation team developed open-ended 
achievement tests, called the Course 1 Posttest and Course 2 Posttest, each in two parts. Part 1 
was designed to be a test of content that both the CPMP and the comparison students would have 
had an opportunity to learn that year, algebraic content for Course 1 and both algebraic and 
geometric content for Course 2. Part 2 of each CPMP Posttest also included subtests of Data 
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Analysis, Discrete Mathematics, Probability, and (in Course 1) Geometry; that is, content that the 
comparison students did not have the opportunity to study. Thus, the comparison students 
completed only Part 1 of the CPMP Posttest at the end of each year, and CPMP students 
completed both parts. These tests required students to construct their responses and to show and 
often explain their work. 

Assessment Administration Schedule 

The paper-and-pencil, mathematics achievement portion of the CPMP field test used a 
pretest-posttest comparison group design for Courses 1 and 2 and pretest-posttest only for 
Course 3. The ATDQT test administered at the beginning of Course 1 served as the pretest for 
all courses, so the pretest-posttest analyses for Courses 1, 2, and 3 are for one, two, and three 
years of mathematics instruction, respectively. Students were allowed unrestricted use of a 
calculator (usually a TI-82 or TI-83) on all achievement tests. The administration schedule for 
the mathematics achievement tests is given in Table 2, and this is followed by a section in which 
the results are presented. 

Table 2. 



Time and Target. Student^ Group for tjie Administrate^ ofEach Achievement Test 





Course 1 
September 


Course 1 
May 


Course 2 
May 


Course 3 
May 


ATDQT 


CPMP & Comp 
(Form K, Lev 15) 


CPMP & Comp 
(FormL, Lev 15) 


CPMP & Comp 
(Form K, Lev 16) 


CPMP 

(FormL, Lev 17/18) 


CPMP Post 
Parti 




CPMP & Comp 


CPMP & Comp 


CPMP 


Part 2 




CPMP 


CPMP 




NAEP Post. 








CPMP 



RESULTS 



The results presented here are mainly quantitative s umm aries of achievement outcomes 
of Core Plus and comparison students across all field test schools and within educationally 
important subsets of students and schools. A broad profile of achievement levels for these 
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subsets of students and schools across several important content areas and types of achievement 
measures is also provided. Specifically, the following results are presented: 

• Overall ATDQT results 

• Results for Various School and Student Groups 

• By School Type (rural, urban, and suburban) 

• By Make-up of CPMP Classes 

• By Gender 

• By First Language and Minority Group Status 

• By High Mathematical Aptitude and Background 

• Results on Various Achievement Outcomes 

• CPMP Posttests 

• NAEP-Based Test 
Overall ATDQT Results 

The results given below were obtained by first converting each student’s raw score to a 
standard score. The standard score is a number that describes the student’s location on an 
achievement continuum, regardless of the ATDQT test form or the students’ grade level. Means 
and other statistics were then computed using the standard scores. As needed for interpretation, 
these summary statistics were then converted to national student or school mean percentiles. 

Norms for the current edition of the ITED were compiled by the test developers in 1992 
using a nationally representative sample of 13,935 high school students. Both student norms 
from the distribution of all students in the norm sample and school mean norms from the 
distribution of all school means in the norm sample are provided by the test publisher. Schoen 
and Ziebarth (1998) have reported results which are based on individual student scores as 
statistical unit. They found that in the 1 1 schools with comparison groups, the Course 1 posttest 
mean of CPMP students was significantly greater than that of the comparison students even 
though the comparison students’ pretest mean was slightly higher. They also found a significant 
school by treatment interaction suggesting that achievement levels of CPMP students compared 
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to those of comparison students differed significantly by school. To account for this interaction, 
results reported here use the school mean on the ATDQT as the statistical unit. 

Results are reported first for two cohort groups, Course 1 and Course 2. ‘In each course, 
the cohort group consists of all CPMP students and all comparison students who completed both 
the ATDQT pretest (given at the beginning of Course 1) and the ATDQT posttest for that course 
(given in May near the end of each school year). Thus, the Course 1 cohort group results are 
indicative of CPMP’s effect in Course 1, and the Course 2 cohort group results show the 
combined effect of Courses 1 and 2. Results for a third group, called the Course 3 cohort group, 
are also presented. The Course 3 cohort group consists of all students who completed the 
ATDQT pretest and the ATDQT posttest at the end of each of the three courses. The numbers of 
students in each cohort group are as follows: 

• Course 1 - 2944 CPMP and 527 comparison 

• Course 2 - 2270 CPMP and 201 comparison 

• Course 3 - 1457 CPMP 

Two of the 33 field test schools, both in areas of transient populations, had five or fewer students 
with complete test data by the end of Course 3. Because of the unreliability of a school mean 
based on so few students, these two schools were excluded from all school analyses that follow 
leaving a total of 3 1 schools in the analysis. 

One measure of the size of a treatment effect in a study with a pretest-posttest design is 
called the effect size which is defined as the difference between the pretest and posttest treatment 
means divided by the standard deviation of the pretest. Thus, the effect size is the number of 
standard deviations (of the pretest) that the mean changed from pretest to posttest. The standard 
deviation of the pretest, rather than posttest, is used because it is the best estimate of the 
variability of the group before it was affected by the treatment. One difficulty with that 
definition for a standardized test like ATDQT is that not all the growth can be attributed to the 
treatment. The norm group also grew from pretest to posttest and that average growth should be 
subtracted from the mean change from pretest to posttest. For a pretest mean at a particular 
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beginning-of-year national percentile p, a good estimate of the posttest mean that reflects the 
average growth of the norm group is the standard score that corresponds to the same end-of-year 
percentile p. For our purposes, this standard score is called the “Norm Mean,” and the following 
definition applies: 



Adjusted Effect Size - Posttest Mean - Norm Mean 
AOjusted trrect bize - pretest Standard Deviation 

The “Adjusted Affect Size,” then, for each cohort by treatment group is the number of pretest 
standard deviations the group grew from pretest to posttest, beyond the average growth of the 
ATDQT norm group. Table 3 summarizes the main results for Course 1 and 2 cohort groups by 
treatment and for the Course 3 cohort of CPMP students. 

Table 3. 

Mean of School Means, Standard Deviation, National School Mean Percentile, and Adjusted 
Effect Size for Pretests and Posttests of the Courses 1, 2, and 3 Cohort Groups by Treatment 




The adjusted effect sizes in the table show that the pretest to posttest gains of the cohorts 
of CPMP students are 26 to .43 standard deviations greater than the norm group’s average gain. 
The gains for CPMP students are also greater than those for comparison students, and gains in 
Courses 2 and 3 are greater than those in Course 1. Concerning the statistical significance of the 
differences in CPMP and comparison group means, a CPMP versus comparison group analysis 
of covariance was conducted for the 1 1 schools with Course 1 comparison groups. With the 
Course 1 Pretest as covariate, the CPMP students’ , adjusted Course 1 Posttest mean was greater 
than that of the comparison students (p = .086). Similarly, an analysis of covariance was 
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conducted for the five schools with Course 2 comparison groups. With the Course 2 cohort’s 
Pretest school means as covariate, the CPMP adjusted Course 2 Posttest mean was greater than 
that of the comparison group (p = .027). 

Another perspective on the pretest to posttest gains in the three cohorts of CPMP students 
is given in Figure 1. These boxplots show the gains in standard scores across the entire 
distribution (minimum, each quartile, and maximum) of CPMP students in each cohort group. 
These plots, from top to bottom in the figure, illustrate the gains associated with one year, two 
years, and three years of CPMP. Numbers given on the plots are national school, mean 
percentiles for the appropriate testing times. 



Course 1 Cohorts 

3 27 



* Co 1 Pretest 
3 



52 70 

n- 



96 



30 



62 



85 



Co 1 Posttest 






97 



Course 2 Cohorts 



h 



34 



68 80 



Co 1 Pretest 



53 



73 84 



Co 2 Posttest 



Course 3 Cohorts 
8 . 



43 



I- 



Co 1 Pretest 



75 84 

DU- 



.98 



14 

I- 



58 



80 88 



4- 



220 



— I— 

240 



Co 3 Posttest 

H 



4- 



97 



99 



.99 



4- 



300 



320 



260 280 

Standard Scores 

Figure 1. Box plots of school means of ATDQT Course 1 pretests and posttests for CPMP 
students in three cohort groups. 

Results for Various School and Student Groups 

By School Type The field test schools included eight rural schools, eight urban 

schools, and 15 suburban schools. ATDQT results are summarized by these school 
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classifications in Table 4. With the exception of urban schools in Course 1, all adjusted effect 
sizes are at least .30. There is no significant difference (p =.05) in posttest means across school 
types for Course 2 and Course 3 cohorts when differences in pretest means are taken into account 
by analysis of covariance. 



Table 4. 

Mean of School Means, National School Mean Percentile, and Adjusted Effect Size for Pretests 
and Posttests of the Courses 1, 2, and 3 Cohort Groups in Rural, Urban, and Suburban Schools 




Interestingly, the effect sizes in urban schools in Courses 2 and 3 were excellent (.62 and 
.59, respectively) even though the students in both those cohort groups were a subset of those in 
the Course 1 cohort group (-.15). According to teachers in urban schools, the students who did 
not continue beyond Course 1 were those with the most problems, whether academic or 
otherwise. The more motivated students who remained were then able to blossom in the 
improved Course 2 and Course 3 classroom environments. Adjusted effect sizes in suburban 
schools, while at or above .30, were somewhat smaller than most of those in rural and in urban 
schools. This may be at least partially due to a statistical effect, regression to the mean, rather 
than to a real difference in CPMP’s effect. Since pretest means in the suburban schools were 
much higher than those in schools of the other types, there is a lower probability of substantially 
still higher scores on the posttest in suburban schools . In other words, for the suburban schools 
there is “less room for growth” on the ATDQT than for the rural and urban schools. 
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By Make-up of CPMP Classes Several different methods of assigning students to 
CPMP classes at the beginning of grade nine were used in the field test schools. Since the CPMP 
curriculum is intended for all (or certainly a very wide range of) students, this variable is a 
potentially important one. There were five main assignment methods. 

• All students: no grouping, whole range of ninth-grade students (5 schools) 

• Range, no top: wide range of prior achievement but excluding best students (13 schools) 

• Wide range: wide range of prior achievement but excluding best and weakest students ( 5 
schools) 

• Coll prep only: more or less the typical Algebra 1 group (6 schools) 

• Work-prep only: more or less the typical general mathematics group (2 schools) 

ATDQT results are summarized by these class assignment methods in Table 5. There is 
no significant difference (p =.05) in posttest means across class assignment method groups for 
any of the three course cohorts when differences in pretest means are taken into account with 



analysis of covariance. 



Table 5. 

Mean of School Means, National School Mean Percentile, and Adjusted Effect Size (Ad ES) for 
Pretests and Posttest of the Courses 1,2, and 3 Cohort Groups by Class Assignment Method 



All students 



% Ad 
M tile ES 



Range, no top Wide range Coll prep only 



% Ad 
M tile ES 



% Ad 
M tile ES 



% Ad 
M tile ES 



Work-prep only 



% Ad 
M tile ES 



CIPr 

ClPo 



261.9 62 
272.4 69 39 



251.8 41 
261.3 49 .17 



263.8 66 

275.8 75 .551 



261.0 60 
272.7 69 .34 




235.4 13 
2383 10 -39 



CIPr 

C2Po 



CIPr 

C3Po 




269.4 75 
293.9 88 .59 



256.9 52 
2783 65 .43 




269.6 76 
288.4 81 .401 




264.1 66 
283.0 73 32 




233.6 12 
257.2 24 1.1 



273.3 81 
3013 88 .37 



259.3 57 
289.7 71 .49 



271.4 77 
296.0 80 .241 



267.8 73 
297.2 82 33 





239.2 18 
277.8 51 13 



There are large fluctuations in results from year to year in some categories, especially 
work-prep. This is at least partially due to the small numbers of schools in the category. The 
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dramatic turnaround after Course 1 in the work prep schools is also an indication that some of the 
students with the worst academic and social problems did not continue into Course 2, as 
discussed above with respect to urban schools. The impressive magnitude of the adjusted effect 
sizes for work prep schools in Course 2 and 3 is probably also partially due to regression to the 
mean, since the pretest means were at the 12th and 18th percentile, respectively. There was a 
great deal of “room to grow” on the ATDQT so improvement was more likely than for groups of 
students whose pretest scores were already well up the scale. With the exception of the work 
prep schools in Course 1, CPMP students’ adjusted effect sizes across the class assignment 
methods and course cohorts were positive and often in the .30 to .55 range. 

By Gender In the CPMP field test sample, there are many interesting gender differences, 
mostly consistent with the compatibility of CPMP with female preferences (Schoen & Pritchett, 
1998). At present, the ATDQT results have been analyzed by gender with little of significance to 
report. Our intent is to complete a careful analysis and report of all data by gender, but that work 
is not complete at this time. 

The following is a sample of ATDQT gender findings. The Course 3 cohort group (N = 
1457) completed not only the Pretest and the Course 3 Posttest, but also the Posttests for Course 
1 and 2. The pattern of growth across all these ATDQT measures by gender, with school by 
gender means as the statistical unit, is given in Table 6. Mean scores for males are significantly 
higher at each test time, but pretest to posttest growth as reflected by the adjusted effect size is 
greater for each test. However, female and male posttest means adjusted for pretest differences 
(using analysis of covariance) are not significantly different at the .05 level for any of the three 
courses. 
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Table 6. 

Mean of School Means, Standard Deviation, National School Mean Percentile, and Adjusted 
Effect Size for Pretests and Posttests of the Course 3 Cohort Groups by Gender 





Female 


Male 




Mean 


S. D. 


%-tile 


Adj. Ef. 
Size 


Mean 


S. D. 


%-tile 


Adj. Ef. 
Size 


Co. 1 Pre 


258.7 


20.7 


55 


liltill 


269.2 


18.2 


75 




Co. 1 Post 


274.0 


20.2 


72 


.43 


280.8 


20.6 


83 


.37 


Co. 2 Post 


283.5 


17.3 


74 


.51 


292.5 


18.1 


99 


.47 


Co. 3 Post 


289.2 


16.5 


70 


.44 


298.2 


16.9 


83 


.23 



By First Language and Minority Group Status In the Course 3 cohort group, there 
were 77 students (5.3%) who indicated that English was not the first language they had learned at 
home before coming to school. The numbers in each school in this category and in the minority 
group categories later in this section were too small to allow the use of school means, so student 
standard scores are the statistical unit for this analysis. Adjusted effect sizes in this section can 
be expected to be lower than those in earlier sections in which school means were the statistical 
unit. The reason is that the denominator of the adjusted effect size, the standard deviation on the 
pretest, is larger because it is a standard deviation of student scores not of school means. In 
general, adjusted effect sizes in this section may be compared to one another but should not be 
compared to those in other parts of the paper. 

With that caveat. Table 7 shows the growth of the two first language groups. The 
adjusted effect sizes for the English as second language (ESL) students are all higher than those 
for the other students suggesting that the CPMP curriculum may be particularly effective with 
such students. Of course, these “ESL” students were not necessarily students with English 
language difficulty at the time that they were in CPMP classes. They simply indicated that 
English was not the first language they had learned at home before coming to school. 
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Table 7. 



Means, Standard Deviation, National School Mean Percentile, and Adjusted Effect Size for 
Pretests and Posttests ofthe Course^ 3 Cohort Groups by non-English First Language 







ESL (N 


= 77) 






Other (N 


= 1,380) 






Mean 


S. D. 


%-tile 


Adj. Ef. 
Size 


Mean 


S.D. 


%-tile 


Adj. Ef. 
Size 


Co. 1 Pre 


261.9 


33.1 


62 


HIM 


268.1 


35.9 


73 




Co. 1 Post 


277.1 


35.9 


77 


.28 


280.8 


38.9 


83 


.18 


Co. 2 Post 


288.9 


29.8 


83 


.37 


289.6 


33.1 


83 


.18 


Co. 3 Post 


292.0 


35.8 


74 


.24 


297.1 


34.4 


82 


.17 



The numbers and percents of Course 3 cohort students who indicated that they were best 
described as members of the following minority groups are: African American (67; 4.6%), 
Hispanic (57; 3.9%), Asian American (45; 3.1%), Native American or Native Alaskan (18; 
1.2%). ATDQT minority group results are presented in Table 8. With two exceptions, all effect 
sizes for all minority groups were at least as high as those of the group of White (non-Hispanic) 
students. For both exceptions (African Americans in Course 1 and Native Americans or Native 
Alaskans in Course 3), the adjusted effect size was very small, but positive, meaning that growth 
was slightly greater than that of the national norm group at the same pretest level. Both these 
minority groups had strong effect sizes in the other two years, so perhaps these near zero effect 
sizes were anomalies. The group which appears to be the most positively impacted by CPMP 
were Hispanics. This finding is consistent with the literature on the social preferences and 
learning styles prevalent among Hispanic groups. 
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Table 8. 

Means, National School Mean Percentile, and Adjusted Effect Size (Ad ES) for Pretests and 
Posttest of the Course 3 Cohort Groups by Minority Group Status 



Cl Pr 
Cl Po 



C2Po 



C3 Po 



Af. Amer. 



% Ad 
M tile ES 



243.0 24 






250.7 25 .02 



265.5 41 .26 



273.6 43 .27 



Hispanic 



% Ad 
M tile ES 



247.3 31 



271.1 67 .45 



280.0 68 .51 



288.5 69 .49 



Asian Amer. 



% Ad 
M tile ES 



277.9 87 



293.9 95 .26 



300.2 93 .20 



310.0 94 .15 



Nat. AmJAl. 



% Ad 
M tile ES 



255.8 50 



271.5 68 .24 



283.3 73 .33 



277.2 49 .01 



White (not His) 



%t Ad 
M ile ES 



269.8 76 i 



282.3 84 .17 



290.9 84 .17 



298.3 83 .15 



By High Mathematical Aptitude and Background When the CPMP evaluation study 
was in its second year, a Mathematics Science Center (MSC) in a medium-sized midwestem city 
requested permission to use the Core-Plus Mathematics curriculum with all their students. This 
school is a magnet school that is for students in grades 9-12 from around the district with 
particularly strong interest, aptitude and background in mathematics and science. 

It was too late to include the MSC in the CPMP field test, but we decided to test the MSC 
students and to follow the progress of the program’s use at the MSC. Thus far, we have 
administered the ATDQT as a pretest at the beginning of Course 1 and in alternative form in 
May as a posttest of Course 1. The following box plot shows the pretest and posttest 
distributions of all 90 MSC students who completed both tests. A perfect score of 40 items 
correct corresponds to a standard score of 353. National percentiles (for the appropriate testing 
time) that correspond to points identified in the box plots are shown on the graph. 
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1 



94 97 99 
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1 \ 1 99 

I 1 

Posttest 

—i 1 1 1 1 1 1 1 1 1 1 1 

250 260 270 280 290 300 310 320 330 340 350 360 

Standard Scores 

Figure 2. Box plot of pretest and posttest distribution for the Mathematics and Science Center 
students 

In spite of a regression to the mean effect due to very high pretest scores, the posttest 
mean was approximately 1 1 standard score points higher than the pretest mean. This is a 
growth about double that of the ninth-grade norm group at the same point in the distribution. 
Viewed another way, the growth of the MSC Core-Plus students’ mean from pretest to 
posttest was about 0.25 standard deviations greater than the growth of the nationally 
representative ninth-grade norm group at the same point in the distribution. 

Results on Various Achievement Outcomes 

CPMP Posttests In order to obtain a measure of students’ attainment of some of 
CPMP’s specific curriculum objectives, the CPMP evaluation team developed posttests for the 
end of each course. These tests, described earlier in this paper, include subtests of various 
content and process outcomes, and performance of CPMP students across these subtests is an 
important part of the CPMP achievement profile. 

Course 1 Posttest Part 1 was comprised of three subtests, called Algebra 
Concepts I, Algebra Concepts II, and Procedural Algebra. The first two subtests required 
students to show that they understood algebraic concepts by applying them in realistic settings 
and interpreting their meaning within those settings. In particular, they were required to translate 
between contextual problem situations and algebraic (linear) representations of the situations 
including graphs, equations or inequalities, and tables. These subtests also required students to 
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re-write algebraic expressions in equivalent forms (that is, simplify expressions and solve 
equations) that provided insights into a problem context, and to explain how solutions or 
equivalent forms represented new information in the problem context. The third subtest required 
students to solve linear equations in one variable and simplify linear expressions with no context. 

A five-point general scoring rubric was used as the basis for developing highly specific 
descriptions of what constituted a score of 0 through 4 on each test item. Scores ranged from 4 
for a “complete, correct response with clear unambiguous work or explanation” to 0 for “no 
response or an irrelevant response.” Graduate and advanced undergraduate Secondary 
Mathematics Education students were trained to use the rubrics to score the posttests. Training 
and practice on the scoring of each task continued until the inter-scorer agreement was 90% or 
higher. 

Figure 3 shows the mean Posttest results for a random sample of 1,102 CPMP students 
and for all 743 students in the traditional comparison classes who completed this test. The 
Cronbach Alpha reliability coefficient for the total test was 0.89. Since CPMP and comparison 
students had almost identical median ATDQT pretest scores (The comparison group had a 
slightly higher mean on the ATDQT pretest.), a comparison of the posttest means is appropriate. 
CPMP students’ CPMP Posttest Part 1 mean scores in these schools were higher than 
comparison students on the Algebra Concepts I and Algebra Concepts II subtests. The effect 
sizes (difference in means divided by the standard deviation of the comparison group) were 0.89 
and 0.59, respectively. On the Procedural Algebra subtest, the comparison group’s mean was 
higher (effect size = 0.22). 
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Algebra Algebra Procedural 

Concepts I Concepts II Algebra 



Figure 3. Course 1 CPMP Posttest Part 1 subtest means of CPMP and comparison groups 
The task in Figure 4, from the Algebra Concepts I subtest, provides as an example of 
the algebraic understanding and reasoning required on the CPMP Posttests. Means and 
standard deviations of the CPMP and comparison groups on each part of this task are also 
given. 



Task 


CPMP 
Mean (SD) 


Comp 
Mean (SD) 


The number of gallons (y) of gasoline left in a large motor boat after 
traveling x miles since filling the tank is given by 

y= 18-2* 






(a). Explain what 18 and -2 in the equation tell about the number of 
gallons. 


2.5 (1.1) 


1.9 (1.2) 


(b). Graph this equation. Explain the role of 1 8 and -2 in the graph. 


2.0 (1.2) 


1.1 (1.0) 


(c). After filling the gasoline tank, Helen drove the boat until there were 
10 gallons left How many miles had she driven? Explain how you 
can tell from the equation and how you can tell from the graph. 


2.2 (1.4) 


1.3 (1.3) 


(d). How many gallons of gasoline were left after Helen had driven the 
boat 8 miles? Show or explain your work. 


2.6 (1.4) 


1.8 (1.6) 


Figure 4. Task from the Course 1 Posttest, Algebra Concepts ] 


’ subtest 





The meaning of the results for parts (a) and (b) is discussed below. Parts (c) and (d) can 
be interpreted in a similar way. In part (a), the intent was for students to indicate that 18 is the 
number of gallons of gasoline the boat had on board at the start and -2 indicates that 2 gallons of 
gasoline are used by the boat for each mile it travels. Such a response was given a score of 4. 
The mean of the CPMP students on part (a) was 2.5, midway between 2 and 3. A score of 3 
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means either (1) that both parts of the question were answered but with some vagueness such as 
“18 is the starting point” or “-2 is the slope” or (2) one question was answered at a 4-level and 
the other was vague or incorrect. The comparison students, mainly from Algebra 1 classes, had a 
mean of slightly less than 2 on part (a). A score of 2 was assigned if ( 1) one part of the response 
was vague but relevant, that is, at the 3-level, but the other part was incorrect or (2) one part of 
the question was answered at the 4-level but the other part was missing. 

In part (b), students were to graph the given linear equation on a grid that was provided. 
An answer at the 4-level would be an accurate graph with an explanation that indicated that 18 
was the y-intercept of the graph and -2 was its slope. The CPMP students’ mean was 2.0, a score 
that was assigned if (1) the graph was accurate but no explanation or a totally irrelevant 
explanation was given or (2) the graph was linear but had mistakes such as a slightly incorrect 
slope or positioning and a relevant but vague explanation was given. On average, the 
comparison students scored at about the 1 -level on this part. This means that the graph was 
incorrect in a serious way such as composed of segments, bars, or a saw tooth or curved shape, 
and no relevant explanation was given. In short, the mean of the comparison students was at a 
level that suggests virtually no understanding of the content that was measured in part (b). 

The Course 2 Posttest Part 1 also contained three subtests, called Coordinate Geometry, 
Algebra Concepts, and Procedural Algebra. The Coordinate Geometry subtest presented a 
contextual situation overlaid on a coordinate system, and students were required to apply 
concepts and methods of coordinate geometry and explain the meaning of the results in the 
context Concepts and methods included finding the equation of a line given two points on it, the 
point of intersection of a vertical line and a second line, the midpoint of a segment, the distance 
between two points, an estimate of the area of an irregular closed region, and the plot of the 
reflection image of a given point across a given line. A related contextual problem situation 
required the use of right triangle trigonometry to solve a triangle for an unknown side. 

As in Course 1, the Algebra Concepts subtest required students to show that they 
understood algebraic concepts by applying them in a realistic setting and interpreting their 
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meaning within that setting. In particular, they were required to translate between problem 
situations and algebraic (in this case, quadratic) representations of the situation including graphs, 
equations or inequalities, and tables. This subtest also required students to transform algebraic 
expressions into equivalent forms (that is, solve equations and simplify expressions) that 
provided insights into a problem context, and to explain how solutions or equivalent forms 
represented new information in the problem context. 

The Procedural Algebra subtest required students to solve linear equations in one variable 
and simplify linear expressions involving parentheses with no context. Students also were 
required to apply the laws of exponents to transform expressions and to decide if given forms 
were equivalent. Each Course 2 Posttest item was scored using a specific rubric that was based 
on the general framework given earlier. 

Figure 5 shows the mean Posttest results for a random sample of 584 CPMP students and 
for all 157 students in the traditional comparison classes who completed this test. The Cronbach 
Alpha reliability coefficient for the total test was 0.86. Since CPMP and comparison students 
had nearly identical median ATDQT pretest scores, a comparison of the pos.ttest means is 
appropriate. (As was true in the Course 1 analysis, the comparison group had a slightly higher 
mean on the ATDQT pretest.) CPMP students’ CPMP Posttest Part 1 mean scores in these 
schools were higher than comparison students on all three subtests. The effect sizes on the 
Coordinate Geometry, Algebra Concepts, and Procedural Algebra subtests were 0.62, 1.27, and 
0.06, respectively. 




Coordinate Algebra Procedural 

Geometry Concepts Algebra 



Figure 5. Course 2 CPMP Posttest Part 1 subtest means of CPMP and comparison groups 
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The task given in Figure 6 is from the Coordinate Geometry subtest. Means and standard 
deviations of the CPMP and comparison groups are provided. Parts (a) - (d) required application 
and some interpretation of usual coordinate geometry methods — finding an equation of a lin e 
through two given points, coordinates of a point on a given line, midpoint of a given segment, 
and distance between two points. The intent in part (e) was for students to use the given grid and 
a valid strategy for estimating the number of square units inside the amusement park. The most 
common valid strategy involved subdividing the park into rectangular and right triangular 
sections, finding the area of each section, and summing the areas. Another productive approach 
was to start with the minimum rectangle that contains the entire park, which has area 33,000 m 2 
and then subtract the areas of regions in that rectangle that are not in the park. Of course, it was 
also necessary to estimate in some reasonable way the areas of regions that were bounded by the 
irregular edges of the park. Whatever estimation strategy was used, the estimate of the area of 
the park should have been more than 20,000 and less than 25,000 m 2 . As a result, students 
should have concluded that the planned park will not have sufficient area to handle the estimated 
crowds. A response with all the above elements received a score of 4. 

The mean of the CPMP students on part (e) was 2.0. A score of 2 was assigned if (1) the 
conclusion was correct and the estimate was between 20,000 and 25,000 but there was no 
explanation or (2) the estimate was between 20,000 and 25,000 yet the conclusion was incorrect 
with an (obviously faulty) explanation. On average, comparison students scored at the 1 -level on 
this part. A score of 1 means that either (1) the estimate was outside the acceptable range and 
there was an explanation or (2) an incorrect conclusion with weak or irrelevant attempts at 
estimation and explanation. 
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Assessment Setting 

A plan for a new Amusement Park is sketched on a grid below. One unit on the grid is equivalent to 10 meters. 
The main gate is located at point G. The entrances to some rides are marked: the Ferris Wheel F, the Roller 
Coaster R, and the Tilt-a- Whirl at T. Complete the following tasks about the plan. 




Task 


CPMP 
Mean (SD) 


Comp 
Mean (SD) 


(a). Main Street is planned to run directly from G to F. Find an equation 
of the line representing Main Street Show or explain your work. 


1.8 (1.4) 


1.0 (0.9) 


(b). The Haunted House H is to be built on Main Street, and it has the 
same x-coordinate as the Roller Coaster. Mark H on the map. To the 
nearest whole numbers, what are the coordinates of H? Show or 
explain your work. 


2.7 (1.2) 


2.1 (1.2) 


(c). A concession stand N is planned midway between the gate and the 
Tilt-a- Whirl. Mark N on the map, and find its coordinates. Show or 
explain your work. 


2.6 (1.2) 


1.9 (0.9) 


(d). The planners want the concession stand to be within 100 meters of 
the roller coaster! Does its present location, found in part (c), satisfy 
this condition? Explain. 


2.5 (1.3) 


2.0 (1.3) 


(e). In order to handle the estimated crowds, the area of the amusement 
park needs to be at least 25,000 m^. Estimate the area of the 
amusement park. Is the area of the planned park enough to handle the 
estimated crowds? Explain how you estimated the area. 


2.0 (1.3) 


1.0 (0.9) 



Figure 6. Task from the Course 2 Posttest, Coordinate Geometry subtest 
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In addition to geometry and algebra strands, CPMP’s curriculum also includes statistics, 
probability, and discrete mathematics. CPMP students were assessed on these two strands on the 



CPMP Posttest Part 2 for each course. Since comparison students would have little opportunity 
to learn this content, they did not complete part 2 of these posttests. An example of a discrete 
mathematics assessment item from Course 2 Posttest Part 2 is given in Figure 7. CPMP 
students’ achievement level in statistics and probability is discussed later. 



Assessment Setting 

For a time in the nineteenth century, a Pony Express mail system was used in 
some of the great plains states. Mail routes between towns were given a 
rating depending on distance, difficulty, and time. Five towns on the Pony 
Express system are St. Joseph (S), Omaha (O), Denver (D), Oklahoma City 
(K), and Dodge City (G). This diagram illustrates the mail routes between 
pairs of these towns and the rating of each. 




Task 


CPMP 
Mean (SD) 


(a). The lower the total rating, the “better” a route. What is the best route from St. Joseph to 
Denver? What is its rating? Explain your reasoning. 


2.6 (1.2) 


(b). A presidential candidate used the mail route ratings to plan a campaign trip from St. Joseph to 
each town and then back to St. Joseph. On the copy of the network below darken a route for 
the campaign trip that has the lowest total rating. What is the total rating of the route? 

S 5 0 

6/ \9 / Vi 2 

g/\ioA\d 

K 


33(1.1) 


(c). This mail system soon proved to be very expensive. 

In order to economize, the U.S. Postal Service S 5 0 

decided to streamline the system. They continued to 6/ \9 /\l2 

operate only those routes that allowed mail to get to ^ J \ 1 0 / V ~ 

\ / .x* D 

every town once in such a way that the total rating 

was minimum. On the right, darken the edges of a ^ 

network that will satisfy these conditions. 


2.6 (1.2) 


Figure 7. Discrete Mathematics task from the Course 2 Posttest Part 2 
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This task is based on content found in the Course 2 unit, “Network Optimization.” Parts 
(a) and (b) involve calculating and comparing ratings for various routes. Part (c) requires a 
minimal spanning tree for this network. Means on the 0 - 4 rubric suggest that most CPMP 
students understood the tasks and completed them successfully. 

NAEP-based Test The National Assessment of Educational Progress (NAEP) is 
administered periodically as a means of monitoring U. S. students’ achievement levels in various 
subject areas. In 1990 and 1992, a NAEP mathematics assessment was administered at several 
grade levels, including grade 12. As another measure of CPMP students’ achievement, a 30-item 
test was constructed with balance among the five content (numbers & operations; measurement; 
geometry; data, statistics & probability; and algebra & functions) and three process categories 
(concepts, problem solving, and procedures). The NAEP subtest and item results are another 
component of the emerging profile of CPMP students’ achievement. 

The NAEP-based test was administered in May 1997 to CPMP students at the end of 
Course 3. A total of 1,292 students in 23 CPMP field test schools completed this test. The 
students in these schools were generally representative of all CPMP students in the field test at 
the end of Course 3. For example, the students in these 22 schools who took both the Course 1 - 
ATDQT pretest and the Course 3 ATDQT posttest had a mean of 270.35 and standard deviation 
of 35.81 on the Course 1 pretest, which is very close to the mean of 267.79 and standard 
deviation of 35.77 for all students in the field test schools. 

The items were part of the NAEP assessment of twelfth-grade students in October of 
either 1990 or 1992. According to Kenney & Silver (1997), the 1992 NAEP sample consisted of 
8,499 students that were representative of all twelfth graders in the country. One important 
descriptor of 1992 twelfth-grade students, from which the NAEP sample was drawn, is the 
highest level of mathematics course taken in high school. This data, taken from school 
transcripts, is given in Figure 8. 
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Alg 2/Geom 
35% 



Pre-calculuj 

28% 




Basic Math 
13 % 



Algebra 

13 % 



Calculus 



11 % 



Figure 8. Percent of 1992 students completing various high school courses (Rock & Pollack, 
1995) 

On the 30-item NAEP-based CPMP test, the mean of the national sample was 12.8 
(42.7%). The Course 3 students’ mean was 16.9 (56.4%) with standard deviation of 5.36. This 
difference in overall means is large (about 0.77 standard deviations), but no assurance can be 
given of the comparability of the two groups’ mathematical aptitudes and backgrounds. (The 30- 
item NAEP-based test had a KR-20 reliability of 0.82 when administered to the CPMP sample.) 
Rather than focus on comparing means of the NAEP sample with those of the Course 3 students, 
the item data from the NAEP sample is used as a benchmark of the difficulty of an item or of all 
items in a content or process category. This approach allows for meaningful between-item and 
between-category comparisons of Course 3 students’ results. 

Given the large difference in overall group means, it is not surprising that the Course 3 
students scored considerably higher than the NAEP sample in all content and process categories. 
However, the magnitude of the differences varied among categories, and it seems reasonable to 
assume that the larger the difference in means in a category, the better the relative performance 
of the CPMP students. Figure 9 illustrates the mean percent correct in each content and process 
category for the Course 3 students and for the NAEP sample. Content and process categories 
appear in descending order of the difference between the two mean percents. 
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CPMP — ■ — National 




Content Subtest Process Subtest 

Figure 9. Percent correct on each NAEP subtest for CPMP and NAEP twelfth-grade sample 

Of the process categories, CPMP students’ performance relative to the NAEP sample was 
best on conceptual items followed by problem solving and finally by items in the procedural 
category. This outcome is consistent with CPMP’s emphasis on sense making, on applications 
and on problem solving with an accompanying de-emphasis on procedural skill practice. 
Although the following presentation of results is organized by content categories, NAEP’s 
process categorization of each sample item is also provided. 

Content Categories Of the five content categories, the Course 3 students scored highest 
on data analysis, statistics and probability, a strand of the CPMP curriculum that is not 
emphasized in most of the more traditional mathematics curricula. Three of the four items in this 
content category were classified by NAEP as conceptual and one as problem solving. All four 
items are given in Figure 10. 
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Data Analysis, Statistics and Probability Item 



CPMP 



NAEP 



[Conceptual] A certain company keeps a list of 50 employees and their annual salaries. 
When the salary of the very highly paid president is added to this list, which of the 
following statistics is most likely to be approximately the same or nearly the same for 
the original list and the new list? 

(a) The highest salary (b) The range (c) The mean 



(d) The median 



(e) The standard deviation 




21 % 



[Problem Solving] From a shipment of 500 batteries, a sample of 25 was selected at 
random and tested. If 2 batteries in the sample were found to be dead, how many dead 
batteries would be expected to be in the sample? 

(a) 10 (b) 20 (c) 30 *(d) 40 (e)50 . 



51% 




93% 



75% 



4 8 12 16 20 24 

Time in Minutes 

[ Conceptual] The total distances covered by two runners during the first 28 minutes of 
a race are shown in the graph above. How long after the start of the race did one runner 
pass the other? 

(a) 3 minutes (b) 8 minutes (c) 12 minutes (d) 14 minutes (e) 28 minutes . 




Age in Years 



51% 



31% 



[Conceptual] In the graph above, each dot shows the number of sit-ups and the 
corresponding age for one of 13 people. According to this graph, what is the median 
number of sit-ups for these 13 people? 

(a) 15 (b)20 {£145 *(d) 50 (e) 55 

Figure 10. Sample data analysis, statistics and probability items 
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On the first item, CPMP students showed an understanding of statistical concepts like 
mean, median, range and standard deviation well beyond the level of the NAEP twelfth-grade 
sample. On the second item, 80% of CPMP students were able to solve a problem involving the 
idea of sampling. The third item simply requires reading a graph, which 93% of CPMP students 
were able to do. The fourth item requires an understanding of a scatterplot and the concept of 
median. CPMP students performed particularly well on the third and fourth items as they 
consistently do on tasks that assess interpretation of information presented graphically. 

Second to data analysis, statistics and probability, CPMP students performed especially 
well on measurement items. The eight items in the measurement category assessed students’ 
understanding of and ability to apply various measurement topics including the volume of a right 
circular cylinder, volume of a rectangular prism, perimeter and area of a rectangle and square, 
and area and circumference of a circle. Six of these items are classified as problem solving and 
two as procedural. Differences in percent correct on the eight items for CPMP students 
compared to the NAEP twelfth-grade sample ranged from 6% to 27%, all higher for the CPMP 
students. Four measurement items are given in Figure 11, including the ones with the greatest 
and least performance difference and one that is classified by NAEP as procedural. 



ERIC 



33 



Achievement in Core-Plus Mathematics 



32 



Measurement Item 


CPMP 


NAEP 


[Problem Solving] The perimeter of a square is 24 centimeters. What is the area of the 
square? 

*(a) 36 cm 2 (b) 48 cm 2 (c) 96 cm 2 (d) 576 cm 2 

(e) I don’t know 


66% 


45% 


[Problem Solving] A rectangular pool 24 feet long, 8 feet wide, and 4 feet deep is filled 
with water. Water is leaking from the pool at the rate of 0.40 cubic foot per minute. At 
this rate, how manv hours will it take for the water level to drop 1 foot? 

Cal 4 *(b) 8 (c) 12 (d) 16 (e) 32 


32% 


26% 


[Problem Solving] In the figure abo> 
inscribed in a square. What is the are 
(a) 3.86 *(b) 7.73 


r n 

/e, a circle wit 
sa of the shade 
(c) 28.27 


h center O and radius of length 3 is 
id region? 

(d) 32.86 (e) 36.00 


64% 


37% 


j7 

[Procedural] The volume V of a rig] 
is given by the formula V = Ttr^h. In 
radius r = 4 and height h = 10? 

(a) 187T (b) 26n (c 


tit circular cylinder like the one in the figure above 
terms of 7T, what is the volume of a cylinder with 

0 80tt *(d) 1607T (e) 1,600;: 


84% 


68% 



Figure 11. Sample measurement items 



On the six items that NAEP classified as algebra & functions, CPMP students averaged 
1 1 .7% higher than the twelfth-grade NAEP sample, third among the five content categories. 
Specific content in this category included evaluating an algebraic expression for a given value of 
x, finding the cosine of an angle, finding the length of an altitude of a right triangle using 
trigonometry, solving an exponential equation, and using a given “if-then” statement to 
determine which of several statements cannot be true. Three of these items were classified as 
conceptual, two as procedural and one as problem solving. Two of the procedural items, a 
conceptual item, and a problem solving item are given in Figure 12. Again, the item 
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performance differences in favor of the CPMP students were less on the procedural items than on 
those classified as conceptual or problem-solving, as was the case for the entire test. 



Algebra & Functions Item 


CPMP 


NAEP 


[Problem Solving] The following statement is true: 

‘If Sally goes to the movie, Mark will go also.” 
Which statement below could NOT be true? 

(a) Sally and Mark both go to the movie. 

(b) Sally goes to the movie and Mark does not go. 

(c) Mark goes to the movie and Sally does not go. 

(d) Neither Mark nor Sally goes to the movie. 

(e) I don’t know. 


63% 


51% 


[ Conceptual] For what value of x is 8^ = 16* ? 

(a) 3 (b) 4 (c) 8 *(d)9 (e) 12 


82% 


34% 


[Procedural) If x = -4, the value of Ax is 

(a) -16 (b)-8 (c) 8 *(d) 16 


83% 


75% 


i 

A 

A 3 

[Procedural] In right triangle ABC a 
*(a)§ (b)f (c 


3 

. 

0 

bove, cos A = 

>5 (d) f (e) ! 


34% 


30% 



Figure 12. Sample algebra & functions items 



The NAEP twelfth-grade sample scored higher than the CPMP students on only two of 
the 30 items on the test. One of those items was in the algebra & functions category. In that 
item, students were to identify a given graph as x > y from among choices x < y, x < y and x > y. 
Just 35% of CPMP students and 36% of NAEP twelfth graders responded correctly. A partial 
explanation for the relatively poor performance of CPMP students on this item may be that the 
curriculum emphasizes a function approach. Thus, CPMP students usually worked with 
inequalities in which y was on the left and f(x) was on the right. 
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On the seven geometry items, the CPMP students average percent correct was 1 1.0 
percentage points higher than that of the NAEP twelfth graders. Three of the seven geometry 
items were classified as conceptual, two as procedural, and two as problem solving. Two 
conceptual, one procedural and one problem solving geometry item are given in Figure 13. 



Geometry Item 


CPMP 


NAEP 


[ Conceptual] Which of the following is NOT a property of every rectangle? 

(a) The opposite sides are equal in length. (b) The opposite sides are parallel, 

(c) All angles are equal in measure. *(d) All sides are equal in length, 

(e) The diagonals are equal in length. 


86% 


71% 


[ Conceptual] The length of a side of t 
radius of the circle? 

(a) 2 *(b) 3 


□ 

he square above is 6. What is the length of the 
(c) 4 (d) 6 ■ (e) 8 


83% 


70% 


> 

[Procedural) What is the slope of the 
(a)f *(b)§ (c)l 


f y, 

3,3) 

line shown in the graph above? 
(d)| (e)3 


47% 


41% 


(Problem Solving) In the xy-plane, a line parallel to the x-axis intersects the y-axis at 
the point (0, 4). This line also intersects a circle in two points. The circle has a radius 
of 5 and its center is at the origin. What are the coordinates of the two points of 
intersection? 

(a) (1, 2) and (2, 1) (b) (2, 1) and (2, -1) (c) (3, 4) and (3, -4) 

*(d) (3, 4) and (-3, 4) (e) (5, 0) and (-5, 0) 


44% 


32% 



Figure 13. Sample geometry items 



Specific content of the seven geometry items included identifying properties of a 
rectangle, finding the slope of a line given the coordinates of two of its points, finding the 
distance between two points with given coordinates, finding the points of intersection of a line 
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and a circle, finding the measure of the angle formed by the bisectors of two adjacent angles, 
finding the length of the side of a square from the radius of the inscribed circle, and applying the 
fact that a parallelogram is formed whenever the midpoints of a quadrilateral are joined in 
succession. 

The five items that were categorized as numbers & operations assessed percent, ratio and 
proportion, and operations with integers. For the most part, these topics are assumed and used in 
applications in both the CPMP and the traditional high school curricula for college-intending 
students but not taught as new content. One topic in this category that is taught in both curricula 
is exponential growth including compound interest. Of the five items in the n umb ers & 
operations category , two were classified as conceptual, two as procedural, and one as problem 
solving. One item of each process type is given in Figure 14. As these items exemplify, number 
& operations was the most difficult of the five content categories with CPMP students avera gin g 
43.7% correct compared to just 34.1% for the NAEP twelfth graders. 



Number & Operations Item 


CPMP 


NAEP 


[Conceptual] Suppose 4t = 3s = 10t, where r, s, and t are positive integers. What is the 
sum of the least values of r, s, and t for which this equality is true? 

(a) 7 (b) 17 *(c) 41 fd) 82 (e) 120 


41% 


30% 


[Procedural) A savings account earns 1 percent per month on the sum of the initial 
amount deposited plus any accumulated interest If a savings account is opened with an 
initial deposit of $1,000 and no other deposits or withdrawals are made, what will be the 
amount in this account at the end of 6 months? 

(a) $1,060.00 *(b) $1,061.52 (c) $1,072.14 (d) $1,600.00 (e) $6,000.00 


36% 


15% 


(Problem Solving) It takes 28 minutes for a certain bacteria population to double. If 
there are 5,241,763 bacteria in this population at 1:00 pm., which of the following is 
closest to the number of bacteria in millions at 2:30 pm on the samp day? 

__(a)80 *(b)40 (cl 20 (d) 15 (e) 10 


41% 


31% 



Figure 14. Sample number & operations items 



The second of two items in which CPMP students had a lower percent correct than the 
NAEP twelfth-grade sa mp le was a number & operations item The stem of this item r ead “In a 
group of 1,200 adults, there are 300 vegetarians. What is the ratio of nonvege tarians to 
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vegetarians in the group?” Only 25% of NAEP twelfth graders and 21% of Course 3 students 
chose the correct response, “3 to 1.” More CPMP students were attracted to “4 to 1” (36.7%) 
and “1 to 4” (31.2%) than to the keyed answer. A partial explanation for the poor performance 
of students on this item may lie in its specialized wording, which is likely to be used when 
students are first learning the ratio concept (that is, in middle school) but is rarely encountered in 
more realistic or mathematically advanced settings. This item was also one of the four poorest 
discriminating items for CPMP students with a discrimination index (biserial correlation of 
scores on the item with scores on all 30 items) of 0.27 compared to a mean item discrimination 
index of 0.40 across all 30 items. 

SUMMARY OF FINDINGS 

In this section, we summarize the findings to date with respect to mathematical 
achievement of CPMP students on various criterion measures. The first set of findings are 
based on ATDQT test forms as criterion measures. 

• When school means are the statistical unit, CPMP students’ posttest means 
(adjusted for pretest differences) were greater than those of a comparison group 
of students enrolled in traditional mathematics curricula at the end of both 
Course 1 (p = .086) and Course 2 (p = .027). 

• Whether aggregated student scores or school means are the statistical uni t, 

CPMP students’ posttest means at the end of Courses 1, 2, and 3 were greater 
than those of the national test norm group at the same pretest levels. 

• With the exception of urban schools at the end of Course 1, the posttest maans 
of CPMP students in rural, urban and suburban schools in Courses 1, 2, and 3 
were greater than those of the national norm group at the same pretest levels. 

• In five categories of CPMP class make-up (all students, wide range but none of 
the very top, wide range but not the very top or very bottom, college-intending 
only, and work-prep only), the posttest means of the CPMP students in Courses 
1, 2, and 3 were greater than those of the national norm group at the samf. 
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pretest levels. One exception was the work-prep only group at the end of 
Course 1. 

• In all three course cohorts, the posttest means of females and the posttest means 
of males in CPMP were both greater than those of the national norm group at 
the same pretest levels. Females grew slightly more than males in each CPMP 
cohort group, but none of the gender differences were statistically significant. 

• In Courses 1, 2, and 3, the posttest means of CPMP students whose first 
language was not English were greater than those of the national norm group at 
the same pretest levels. 

• In Courses 1, 2, and 3, the posttest means of CPMP students in all minority 
groups (African Americans, Asian Americans, Hispanics, and Native/ Alaskan 
Americans) were greater than those of the national norm group at the same 
pretest levels. Hispanics made the greatest pretest to posttest gains at the end of 
each course. 

• The Course 1 Posttest mean of CPMP students in a Mathematics and Science 
Center was about double that of the national norm group at the same pretest 
level. (This group’s median pretest score was at the 97th national percentile.) 

The next set of findings are based on CPMP Posttests for Courses 1 and 2 as criterion 
measures. 

• CPMP students scored significantly higher than comparison students on subtests 
of algebraic concepts (Courses 1 and 2) and of geometric concepts (Course 2). 

• At the end of Course 1, comparison students scored higher than CPMP students 
on a subtest of algebraic procedures, but the CPMP mean was slightly higher 
than the comparison mean at the end of Course 2. 

• CPMP students also illustrated an understanding of statistics and probability 
topics (e.g. centers and variability of distributions, various ways to display and 
interpret data graphically, simulation, and expected values in Course 1 and 
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correlation, least squares regression, waiting time distributions, independent 
events, and fair price in Course 2) and of discrete mathematics topics (e.g. Euler 
paths, critical paths, and graph coloring in Course 1 and minimal spanning trees 
and matrices in Course 2). 

The following findings are based on the NAEP-based test administered at the end of 
Course 3 as criterion measure. 

• Relative to the nationally representative NAEP sample of twelfth-grade 
students, CPMP students scored much better on each content subtest with the 
mean differences ordered from largest to smallest as follows: Data, Probability 
& Statistics; Measurement; Algebra & Functions; Geometry; and Numbers & 
Operations. 

• Relative to the nationally representative NAEP sample of twelfth-grade 
students, CPMP students scored much better on each process subtest with the 
mean differences ordered from largest to smallest as follows: Concepts, 

Problem Solving, and Procedures. 

FINAL NOTES 

The CPMP field test included schools from a broad range of community environments 
and with diverse student bodies. Students in CPMP courses after one year, two years, and three 
years illustrated better understanding of, and ability to reason in, quantitative situations (as 
measured by ATDQT) than did students in more traditional mathematics courses and in the 
nationally representative norm group. Course 1 and Course 2 students were also better able to 
reason with, and apply, the concepts and methods of algebra and geometry that were measured 
by the CPMP Posttests. At the end of Course 1, comparison students scored higher on a subtest 
of algebraic procedures, but by the end of Course 2 this difference had disappeared. At the end 
of Course 3, CPMP students performed particularly well on NAEP-based measures of data 
analysis, probability and statistics and on measures of conceptual understanding. Their 
performance was somewhat lower in some other content areas and on items assessing procedural 
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outcomes, but still considerably higher than a nationally representative sample of twelfth-grade 
students. 

CPMP students achieved well whether they were in rural, urban or suburban schools. 
They also achieved well in a variety of grouping arrangements including all students, wide range 
but none of the very top, wide range but not the very top or very bottom, college-intending only, 
and work-prep only. Both female and male students showed good achievement in CPMP. In 
CPMP, students whose first language was not English achieved at least as well as other students. 
Students in several minority groups (African American, Asian American, Hispanic, and 
Native/Alaskan American) varied in their pretest scores, but their pretest to posttest growth was 
solid and in line with that of white, non-Hispanic students. Hispanic students’ growth was 
particularly strong. There is also evidence from a Mathematics and Science Center that the 
CPMP curriculum can work well with students of very high ability. 

To re-emphasize an important point made at the beginning of this paper, the profile of 
achievement presented is an emerging one in the sense that CPMP’s evaluation is ongoing and 
will continue for several more years. ACT and SAT data from the Course 3 field test are still 
being processed and analyzed. The Course 4 field-test students will complete several 
achievement measures. These students, the class of 1999, will be the first group to graduate from 
high school having completed four years of the field-test versions of the CPMP curriculum. Data 
concerning what happens to these students after high school especially with respect to their 
mathematical preparedness will eventually be a part of the envisioned profile. 

The findings reported in this study are likely the result of CPMP students engaging 
frequently in class, homework, and assessment activities that provide them ample opportunity to 
reason about problems presented in realistic contexts, about mathematical models for those 
problems, and about connections among and patterns in various representations of those models. 
Class observation and teacher perception data, while still being analyzed, provide further 
evidence in support of the above explanation for the positive CPMP student achievement 
outcomes. Furthermore, analysis of students’ perceptions of their experience in CPMP classes 
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revealed two themes that were consistently rated very positively: (1) solving realistic and 
challenging problems is difficult, especially at first, but later students gain from the experience 
and find it interesting; and (2) problem solving in groups, with the accompanying discussing and 
writing of mathematical ideas, is an important aid to learning (Schoen & Ziebarth, 1997). 

Related to these themes, many teachers and students have also expressed the belief that 
mathematical sense-making in contextual settings is an aid to memory. 

A conjecture in need of more research is suggested by the fairly dramatic improvement of 
urban students and of African American students in Courses 2 and 3 after slow starts in Course 1. 
The students in these groups who remained in the CPMP curriculum and continued to work at 
doing mathematics, improved in their performance, and were successful. It appears that, for 
them, tenacity and hard work paid off. The importance of effort, so highly valued in 
mathematically high-achieving countries, seems to be an important determiner of achievement in 
a core mathematics curriculum. While this and other important researchable issues remain in 
need of investigation, the emerging profile of achievement at its present stage suggests that good 
implementation of the CPMP curriculum results in solid outcomes on various achievement 
measures and for a wide range of secondary school students in a wide variety of schools. 
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